Hybrid PSO-SA algorithm for training a Neural Network for Classification

In this work, we propose a Hybrid particle swarm optimization-Simulated annealing algorithm and present a comparison with i) Simulated annealing algorithm and ii) Back propagation algorithm for training neural networks. These neural networks were then tested on a classification task. In particle swarm optimization behaviour of a particle is influenced by the experiential knowledge of the particle as well as socially exchanged information. Particle swarm optimization follows a parallel search strategy. In simulated annealing uphill moves are made in the search space in a stochastic fashion in addition to the downhill moves. Simulated annealing therefore has better scope of escaping local minima and reach a global minimum in the search space. Thus simulated annealing gives a selective randomness to the search. Back propagation algorithm uses gradient descent approach search for minimizing the error. Our goal of global minima in the task being done here is to come to lowest energy state, where energy state is being modelled as the sum of the squares of the error between the target and observed output values for all the training samples. We compared the performance of the neural networks of identical architectures trained by the i) Hybrid particle swarm optimization-simulated annealing, ii) Simulated annealing and iii) Back propagation algorithms respectively on a classification task and noted the results obtained. Neural network trained by Hybrid particle swarm optimization-simulated annealing has given better results compared to the neural networks trained by the Simulated annealing and Back propagation algorithms in the tests conducted by us.